Abstract
Recent advances in EEG technology makes brain-computer-interface (BCI) an exciting field of research. BCI is primarily used to adopt with the paralyzed human body parts. However, BCI in envisioned speech recognition using electroencephalogram (EEG) signals has not been studied in details. Therefore, developing robust speech recognition system using EEG signals was proposed. In this paper, we propose a coarse-to-fine-level envisioned speech recognition framework with the help of EEG signals that can be thought of as a serious contribution in this field of research. Coarse-level classification is used to differentiate/categorize text and non-text classes using random forest (RF) classifier. Next, a finer-level imagined speech recognition of each class has been carried out. EEG data of 30 text and not-text classes including characters, digits, and object images have been imagined by 23 participants in this study. A recognition accuracy of 85.20 and 67.03% has been recorded at coarse- and fine-level classifications, respectively. The proposed framework outperforms the existing research work in terms of accuracy. We also show the robustness in envisioned speech recognition.
Similar content being viewed by others
References
Brigham Katharine, Vijaya Kumar BVK (2010) Imagined speech classification with EEG signals for silent communication: a preliminary investigation into synthetic telepathy. In: 4th international conference on bioinformatics and biomedical engineering, pp 1–4
Pineda JA, Allison BZ, Vankov A (2000) The effects of self-movement, observation, and imagination on/spl mu/rhythms and readiness potentials (RP’s): toward a brain-computer interface (BCI). IEEE Trans Rehabil Eng 8 (2):219–222
Jara AJ, Lopez P, Fernandez D, Castillo JF, Zamora MA, Skarmeta AF (2014) Mobile discovery: discovering and interacting with the world through the internet of things. Pers Ubiquit Comput 18(2):323–338
Han K, Kim J, Shon T, Ko D (2013) A novel secure key paring protocol for rf4ce ubiquitous smart home systems. Pers Ubiquit Comput 17(5):945–949
Metsis V, Kosmopoulos D, Athitsos V, Makedon F (2014) Non-invasive analysis of sleep patterns via multimodal sensor input. Pers Ubiquit Comput 18(1):19–26
Pei X, Hill J, Schalk G (2012) Silent communication: toward using brain signals. IEEE Pulse 3(1):43–46
Kaur B, Singh D, Roy PP A novel framework of EEG-based user identification by analyzing music-listening behavior. Multimedia Tools and Applications 1–22. https://doi.org/10.1007/s11042-016-4232-2
Badcock NA, Mousikou P, Mahajan Y, de Lissa P, Thie J, McArthur G (2013) Validation of the emotiv EPOC®; EEG gaming system for measuring research quality auditory ERPs. PeerJ 1:e38
Kumar P, Saini R, Roy PP, Dogra DP (2017) A bio-signal based framework to secure mobile devices. J Netw Comput Appl 89:62–71
Gandhi T, Panigrahi BK, Anand S (2011) A comparative study of wavelet families for EEG signal classification. Neurocomputing 74(17):3051–3057
Matsumoto M, Hori J (2014) Classification of silent speech using support vector machine and relevance vector machine. Appl Soft Comput 20:95–102
Houde JF, Nagarajan SS, Sekihara K, Merzenich MM (2002) Modulation of the auditory cortex during speech: an MEG study. J Cogn Neurosci 14(8):1125–1138
Price CJ (2012) A review and synthesis of the first 20years of PET and fMRI studies of heard speech, spoken language and reading. Neuroimage 62(2):816–847
Kanjo E, Al-Husain L, Chamberlain A (2015) Emotions in context: examining pervasive affective sensing systems, applications, and analyses. Pers Ubiquit Comput 19(7):1197–1212
Peng H, Bin H, Zheng F, Fan D, Zhao W, Chen X, Yang Y, Cai Q (2013) A method of identifying chronic stress by EEG. Pers Ubiquit Comput 17(7):1341–1347
Menezes MLR, Samara A, Galway L, SantAnna A, Verikas A, Alonso-Fernandez F, Wang H, Bond R (2017) Towards emotion recognition for virtual environments: an evaluation of EEG features on benchmark dataset. Pers Ubiquit Comput 1–11. https://doi.org/10.1007/s00779-017-1072-7
Costa EJX, Cabral EF (2000) EEG-based discrimination between imagination of left and right hand movements using adaptive gaussian representation. Med Eng Phys 22(5):345–348
DaSalla CS, Kambara H, Sato M, Koike Y (2009) Single-trial classification of vowel speech imagery using common spatial patterns. Neural Netw 22(9):1334–1339
Parra LC, Spence CD, Gerson AD, Sajda P (2003) Response error correction-a demonstration of improved human-machine performance using real-time EEG monitoring. IEEE Trans Neural Syst Rehabil Eng 11 (2):173–177
D’Zmura M, Deng S, Lappas T, Thorpe S, Srinivasan R (2009) Toward EEG sensing of imagined speech. In: In International Conference on Human-Computer Interaction, pp 40–48
Li W, Zhang X, Zhong X, Zhang Y (2013) Analysis and classification of speech imagery EEG for BCI. Biomed Signal Process Control 8(6):901–908
Hsu Y-L, Yang Y-T, Wang J-S, Hsu C-Y (2013) Automatic sleep stage recurrent neural classifier using energy features of EEG signals. Neurocomputing 104:105–114
He SL, Gao X, Yang F, Gao S (2003) Imagined hand movement identification based on spatio-temporal pattern recognition of EEG. In: 1st EMBS conference on neural engineering, pp 599– 602
Deng S, Srinivasan R, Lappas T, D’Zmura M (2010) EEG classification of imagined syllable rhythm using hilbert spectrum methods. J Neural Eng 7(4). https://doi.org/10.1088/1741-2560/7/4/046006
Esfahani ET, Sundararajan V (2012) Classification of primitive shapes using brain–computer interfaces. Comput Aided Des 44(10):1011–1019
Torres-García AA, Reyes-García CA, Villaseñor-Pineda L, García-Aguilar G (2016) Implementing a fuzzy inference system in a multi-objective EEG channel selection model for imagined speech classification. Expert Systems with Applications 59:1–12
González-Castañeda EF, Torres-García AA, Reyes-García CA, Villaseñor-Pineda L (2017) Sonification and textification: Proposing methods for classifying unspoken words from EEG signals. Biomed Signal Process Control 37:82–91
Wang K, Wang X, Li G (2017) Simulation experiment of bci based on imagined speech EEG decoding. arXiv:1705.07771
Nguyen CH, Karavas G, Artemiadis P (2017) Inferring imagined speech using EEG signals: a new approach using riemannian manifold features J Neural Eng https://doi.org/10.1088/1741-2552/aa8235
Soleymani M, Pantic M, Pun T (2012) Multimodal emotion recognition in response to videos. IEEE Trans Affect Comput 3(2):211–223
Gauba H, Kumar P, Roy PP, Singh P, Dogra DP, Raman B (2017) Prediction of advertisement preference by fusing EEG response and sentiment analysis. Neural Netw 92:77–88
Donos C, Dümpelmann M, Schulze-Bonhage A (2015) Early seizure detection algorithm based on intracranial EEG and random forest classification. Int J Neural Syst 25(05):1550023
Fraiwan L, Lweesy K, Khasawneh N, Wenz H, Dickhaus H (2012) Automated sleep stage identification system based on time–frequency analysis of a single EEG channel and random forest classifier. Comput Methods Prog Biomed 108(1):10–19
Menze BH, Kelm BM, Masuch R, Himmelreich U, Bachert P, Petrich W, Hamprecht FA (2009) A comparison of random forest and its Gini importance with standard chemometric methods for the feature selection and classification of spectral data. BMC Bioinf 10(1):213
Breiman L (2001) Random forests. Mach Learn 45(1):5–32
Übeyli ED (2009) Combined neural network model employing wavelet coefficients for EEG signals classification. Digital Signal Process 19(2):297–308
Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1–3):131–159
Yadava M, Kumar P, Saini R, Roy PP, Dogra DP (2017) Analysis of EEG signals and its application to neuromarketing. Multimedia Tools and Applications 76(18):19087–19111
Kumar P, Gauba H, Roy PP, Dogra DP (2017) A multimodal framework for sensor based sign language recognition. Neurocomputing 259:21–38
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kumar, P., Saini, R., Roy, P.P. et al. Envisioned speech recognition using EEG sensors. Pers Ubiquit Comput 22, 185–199 (2018). https://doi.org/10.1007/s00779-017-1083-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00779-017-1083-4